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^^Method and apparatus for monitoring operation of 
processing systems, related network and computer 
program product therefor" 

* * * 

5 Field of the invention 

This invention relates to techniques for 
monitoring (e.g. analyzing) operation of processing 
systems such as computer systems and networks . 

The invention was developed by paying specific 

10 attention to the possible application to computer 
intrusion detection systems, i.e. systems^ that detect 
security problems in computer systems ':and networks 
caused by the malevolent action of an external or 
internal agent . The agent can be an automatic system 

15 (i.e. a computer virus or a worm) or a human intruder 
who tries to exploit some weaknesses in the system for 
a specific purpose (i.e. unauthorized access to 
reserved data) . 

Description of the related art 

20 The purpose of a computer intrusion detection 

system (IDS) is to collect and analyze information on 
the activity performed on a given computer system in 
order to detect, as early as possible, the evidence of 
a malicious behavior. 

25 Two fundamental mechanisms have been developed so 

far in the context of intrusion detection, namely: 
network-based intrusion detection systems (i.e. so- 
called NIDS) and host -based intrusion detection systems 
(HIDS) . 

30 NIDS analyze packet flow in the network under 

surveillance, searching for anomalous activities; the 
vast majority of NIDS employs pattern-based techniques 
to discover evidence of an attack. Conversely, HIDS 
operate on a per-host basis, using a wider variety of 

35 techniques, to accomplish their purpose. 
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HIDS are usually better tailored for detecting 
attacks likely to really impact on the host under their 
control . 

NIDS systems have a broader vision over the 
5 computer network than their host -based counterpart; so 
they can correlate different attacks more easily and 
can detect anomalies that can be neglected if only a 
single host is taken into account. However some 
specific attacks that involve ciphered connections or 
10 some form of covert channels, are extremely harder to 
discover using only network based techniques. As a 
consequence, both approaches must be preferably be used 
in a truly complete and effective intrusion detection 
system. 

15 Two fundamental figures are currently evaluated in 

order to measure the effectiveness of an intrusion 
detection system: the rate of false-positives and the 
rate of false-negatives. False-positives designate 
those normal situations that are erroneously detected 

20 as attacks, false-negatives are effective attacks which 
are not correctly identified by the IDS. 

The primary goal of an IDS is to minimize these 
figures, while maintaining an acceptable analysis rate 
(that is, the number of events that can be analyzed in 

25 the time unit) . 

Obviously, different technologies result in 
different false-positive and false -negative rates. The 
most common techniques employed in intrusion detection 
systems are misuse detection and anomaly detection. 

30 Artificial intelligence and state analysis techniques 
have been used occasionally in few implementations. 

Misuse detection is the technique commonly adopted 
in NIDS. Usually, some sort of pattern matching 
algorithm is applied over a series of rules to detect 

35 misuse conditions. This approach is discussed^ for 
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example, in ^^A Pattern Matching Model for Misuse 
Intrusion Detection" by S. Kumar, E. Spafford and al . 
in the Proceedings of the 17^^ National Computer- 
Security Conference. Also, several patents have issued 
5 in connection with pattern-based IDS systemai, US-A-B 
278 901 and US-A-6 487 666 being cases in point. 

Specifically, US-A-5 278 901 discloses an 
intrusion detection technique based on state analysis. 
The prior art document in question describes several 

10 independent intrusion patterns using the graph 
formalism, and provides a mechanism that, starting from 
the audit trail generated by the host operating system, 
is able to detect whether a sequence of operations on 
the system can be mapped onto one of the graphs 

15 .representing the intrusion scenarios. The complexity 
involved in defining the patterns that model an 
intrusion, makes this approach unsuitable for use in 
anomaly-based intrusion detection systems. 

More generally, pattern-based systems are well 

20 suited for NIDS but are not very efficient in the 
context of HIDS as they can generate high false- 
negative rates: in fact HIDS fail to detect something 
for which a specific signature has not been provided. 

Anomaly detection has also been widely used for 

25 both network-based and host-based intrusion detection 
systems (especially with HIDS) . When such an approach 
is resorted to, the IDS is trained (using a pre-defined 
policy or some foarm of automatic learning) on the 
normal system behavior, and detects any deviation from 

30 this standard configuration. Clearly, this approach is 
able to cope with unseen attack patterns, reducing the 
false-negative rate. However, it also shows a markedly 
higher false-positive rate, because some permitted 
actions have not been included in the policy or have 

35 not been observed during the learning stage. For a 
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detailed discussion of the application of anomaly 
detection in the field of intrusion detection, 
reference can be made to D. Wagner and D. Dean:- 
""Intrusion Detection Via Static Analysis/', IEEE 
5 Symposium on Security and Privacy, 2001. 

One of the most interesting applications of 
anomaly detection in the context of host -based IDS is 
the analysis of the sequences of system calls, or 
system primitives, issued during the normal process 

10 activity. Generally speaking, a modern operating system 
uses at least two different levels of j^rivilege for 
running applications: at the user * level, the 
application behavior is constrained and the single 
application cannot manipulate arbitrarily system wide 

15 resources, while at the kernel level, the application 
has a complete control over the system. The transition 
between the user and the kernel levels is regulated by 
the system calls, also known as ^'syscalls'' or system 
primitives, which allow an non- trusted application to 

20 manipulate a system-wide resource. For example, using a 
system call an application can spawn (or terminate) 
another application, create a file, or establish a 
network connection. 

The possibility of monitoring effectively the 

25 behavior of a given application is ' broadly accepted; 
for example, S. Forrest and al . in ^'A Sense of Self for 
Unix Processes" published in the 1996 IEEE Symposium on 
Security and Privacy, discuss a method for anomaly 
detection based on the short-range correlation of 

30 sequences of system calls. 

In EP-A-0 985 995, an advanced application of this 
technique is disclosed which relies on the TEIRESIAS 
algorithm. The arrangement of EP-A-0 985 995 uses 
system call analysis to derive a complete 

35 characterization for a given application; this .means 
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that for the specific process executing an instance of 
the application, the entire sequence of system calls is 
collected and organized in sequence of repeated- 
patterns; at detection time, a sequence of system call 
5 is then compared with the list of pattern to^. identify 
an anomaly in the application. 

In US Patent Application US-2002/013 8755 another 
intrusion detection technique is discussed based on 
anomaly detection. The disclosure focuses on a method 

10 that allows modeling the behavior of a single process 
using a set of logical formulas, known as Horn Clauses . 
These formulas are derived from both acceptable and 
dangerous sequences of system calls, which are used to 
describe some exemplary behavior, although the 

15 algorithm can produce a working model using only 
acceptable sequences . 

System call analysis is not only suitable for 
anomaly detection approaches, but it can be used also 
in a more classical misuse detection scheme. 

20 For instance U.S. Patent applications US- 

2002/0083343 and US~2002/0046275 disclose an intrusion 
detection system architecture that exploits, among 
other sources of data, system call events to detect 
sequences that can possibly indicate an intrusion. The 

25 architecture described in these prior art documents is 
fairly complex and it is based on various modules and 
layers to build a comprehensive IDS system. 

State analysis techniques are another emerging 
area of research in the field of intrusion detection. 

30 These techniques have been the main focus of the 

so-called ""STAT" project, which is discussed in various 
papers such as K. Ilgun, R. A. Kemmerer, and P. A. 
Porras, "State Transition Analysis: A Rule-Based 
Intrusion Detection Approach", IEEE Transaction on 

35 Software Engineering and G. Vigna, S. T. Eckmann and R. 
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A. Kemmerer ''The STAT Tool Suite'', Proceedings of 
DISCEX 2000. The STAT framework is fairly generic, and 
defines a language to express and represent the attack- 
scenarios for different kinds of context (i.e. both in 
5 the NIDS and HIDS contexts) . Specifically, some 
specific attack scenarios are defined that have to be 
matched on the system model for the attack to be 
detected (therefore the system performs a misuse 
detection) . Moreover these attack scenarios have to be 

10 explicitly coded in some computer-based language. 
Object and summary of the invention ^ 
The basic object of the present invention is thus 
to provide an improved arrangement that dispenses with 
with the intrinsic drawbacks of the prior art 

15 . extensively considered in the foregoing. Specifically, 
the present invention aims at : 

dispensing with the disadvantages of those 
arrangements based on misuse detection that are exposed 
to the risk of generating a high number of ''false 

20 negatives" if the rules that dictate operation of the 
system are not continuously and timely updated, and 

- providing, in the case of arrangements based on 
anomaly detection, system-wide operation, without 
limitations to any specific application and making it 

25 possible for the arrangement to become an expert system 
adapted to learn proper intervention policies - 

According to the present invention, that object is 
achieved by means of a method having the features set 
forth in the claims that follow. The invention also 

30 relates to a corresponding apparatus, a related network 
as well as a related computer program product, loadable 
in the memory of at least one computer and including 
software code portions for performing the steps of the 
method of the invention when the product is run on a 

35 computer. As used herein, reference to such a computer 
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program product is intended to be equivalent to 
reference to a computer-readable medium containing 
instructions for controlling a computer system to^ 
coordinate the performance of the method of the 
5 invention. Also, reference to at least one computer is 
intended to highlight the possibility for the invention 
to be implemented in a de-centralized fashion. 

A preferred embodiment of the invention is thus an 
anomaly based monitoring system which exploits a 

10 mixture of the state analysis technique, "syscall" 
(system primitives) sequence analysis and rule -based 
reasoning. Specifically, such a preferred, embodiment 
provides for monitoring operation of a processing 
system including a set of system resources and having a 

15 .plurality of processes running thereon by monitoring 
operation of a set of primitives. The set of primitives 
monitored is selected as a set comprised of system 
primitives that i) allocate or release said system 
resources, and ii) are used by different processes in 

20 said plurality. 

Such a preferred embodiment of the invention is 
based on the recognition that a processing system 
including a set of system resources can be effectively 
monitored, e.g. for intrusion detection purposes, by 

25 achieving system-wide operation by monitoring a set 
primitives used by different processes (and not just by 
a single application) . The related processing load may 
be maintained within reasonable limits by selecting the 
primitives in question as system primitives that 

30 allocate (i.e. request) or release one of the system 
resources . 

Preferably, the set of primitives monitored 
includes all the system primitives that allocate or 
release said system resources or includes exclusively 
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those system primitives that allocate or release said 
system resources . 

A preferred embodiment of the arrangement of the- 
invention is a system comprising three high-level 
5 logical components, namely: 

- a system-wide information gathering component, 
which intercepts low- level data from the host system, 
and allows watching every change in the state of the 
system, while providing data to be analyzed for 

10 monitoring purposes, e.g. in order to detect 
intrusions; low-level data comprises i^ystem calls, or 
system primitives, with their call and return 
parameters, and information relative to system 
resources in use (e.g. file, socket, device..); 

15 - a detection component, which represents the core 

of the monitoring system, carries out the data 
analysis. It performs anomaly detection by revealing 
differences between the current state of the system and 
the state recorded during a previous period of time 

20 when the system is assumed to be safe. These anomalies 
consist in suspicious events that could represent an 
intrusion, so they can cause the emission of an alert 
to a management system; and 

- a management system, which shows all the alerts, 
25 collects them for off-line analysis and possibly 

generates graphical reports. Moreover, it allows the 
administrator to tune and configure the whole system. 

The detection component preferably includes three 
logical siib- components with specific goals. The first 

30 sub-component maintains a real-time high-level model of 
the current state of the monitored host. The second 
sub-component is comprised of different modules that 
use that high-level model to perform the anomaly 
detection, each of them having a specific view of the 

35 whole system, such as network activity or file system 
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Status. The third sub^component receives and correlates 
the anomalies to decide if they can represent an 
intrusion and, in this case, issues an alert to the- 
managemen t s ys t em . 
5 The basic idea of the arrangement described • herein 

is to build a synthetic, comprehensive representation 
of the system; this model has to be initialized 
correctly, in order to reflect the current state of the 
system when the intrusion detection system is started . 
10 After initialization, each security-related event 
generated in the real system is forwarded to the model, 
which is updated accordingly. Usually such -events come 
in the form of well-defined system calls. A specific 
component of the intrusion detection system (running in 
15 kernel space) selects the system calls that need to be 
analyzed and forwards them to the user space component . 

In that way, even if decoupled, the model and the 
real system remain perfectly synchronized. 

An intrusion detection system may thus perform 
20 specific analysis on the system model, tracking various 
kinds of anomalies. The system is built using a modular 
approach, so it is possible to extend and tailor the 
configuration according to the characteri'stics of the 
host under surveillance. 
25 Preferably, several different analysis modules 

(called "knowledge bases") are implemented. An 
application knowledge base tracks the processes that 
run on the system, monitoring the resource they use and 
their current state. A file- system knowledge base 
30 controls all the file related operations, such as 
creation and deletion of files, and so on. A network 
knowledge base analyzes all the incoming and outgoing 
connection, searching for anomalous activities. 

Each knowledge base can operate in two essential 
35 modes; in the learning mode, it updates itself 
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accordingly to the events occurring in the system; in 
the analysis mode, it compares the current state of the 
system with the state observed during the learning- 
stage. The main purpose of knowledge bases is to 
5 emulate the way in which a human system admiiiist rater 
detect that something wrong is happening on the system. 
More precisely, the knowledge bases provide an analog 
model of a system sub-component which is close to the 
model that a human system administrator keeps in mind 

10 when searching the system for anomalies . 

Whenever a knowledge base detects ^an anomalous 
behavior, a signal (hereinafter defined "I'ed alert") is 
raised; each led alert is assigned a specific weight, 
defined by the knowledge base itself. The weight gives 

15 , an indication about the criticality of the event . For 
example, the weight assigned to the execution of a 
completely new application is higher than the weight 
assigned to the execution of an extra instance of an 
already known application that has never been launched 

20 twice simultaneously by the same user. 

All the led alerts are collected by an alerter 
module, which uses a rule based mechanism to correlate 
and aggregate this information. For example, if a new 
application is activated, a new file is created from 

25 this specific application and a new network connection 
is activated; all these events generate led alerts that 
are aggregated in a single user- level alert that 
pinpoints what is happening on the target host . 
Moreover, a damping mechanism is used to avoid that an 

30 overwhelming number of signals may produce a long 
sequence of identical alerts. 

The alerter module employs different algorithms to 
process and analyze alerts. Some specific sequence of 
action can be easily mapped onto ''bad behaviors" ; for 

35 other scenarios, it is possible to detect some general 
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anomalies that the operator needs to further track 
down. 

A further element in the preferred system- 
architecture described herein is a management system. 
5 This is used to produce alerts in a human readable form 
and archive them for forensic purposes or trend 
analysis . 

The arrangent described herein thus uses an 
anomaly detection scheme based on the analysis of the 

10 system call events generated by the entire system, that 
is those system primitives that either allocate or 
release one of the system resources . 

The stream of events thus monitored is used to 
build a synthetic representation of the system; the 

15 analysis is then conducted on this optimized 
representation, using specific modules, which address 
well defined areas of the system. For example, a 
process analysis module is used to examine all the 
application running on the system, monitoring some 

20 specific actions; a file-system module is used to 
analyze the operations at the file level . Whenever a 
module detects something that has never been observed 
in the past, it generates a signal. All the signals are 
collected by a correlation engine that, using a 

25 specific rule-base, decides whether to emit a console 
alert or not . 

To sum up, a preferred embodiment of the invention 
uses a combination of system call analysis, state-based 
analysis and rule-based reasoning to detect attacks on 

30 the controlled host- The selection of the specific 
system calls to be used for that purpose and the way 
data are collected play a significant role in such an 
arrangement. Another significant feature is the 
organization of the various knowledge bases; each 

35 knowledge base defines a specific way to look at the 
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system, just like a human administrator would do. The 
knowledge base captures the set of tests and analyses 
that a skilled administrator performs on the system to- 
detect the early signs of an anomaly. The anomaly 
5 detection paradigm is thus used to 1 earn" ^. what is 
considered to be the normal system behavior; the system 
state is then built from the learning stage and the 
intrusion detection is performed by searching for 
relevant anomalies from the regular system state. 

10 It will be appreciated that a major difference 

between the arrangement discussed in EP-A-O 985 995 and 
a preferred embodiment of the present invention is the 
use of formal logic to model the system behavior 
compared to the analog models used in a preferred 

15 embodiment of the current invention. The analog models 
mimic the effective system component under analysis, 
providing a simplified view on it, which is essentially 
similar to the reference picture used by the 
administrator to track down any system anomaly. This 

20 model is somewhat ^^fuzzier'' than the one used in the 
prior art, but is able to capture a broader view on the 
system under analysis. 

The arrangement described herein is 'thus adapted 
to provide improved operation and results, while 

25 dispensing with the disadvantages of those arrangements 
based on misuse detection that are exposed to the risk 
of generating a high number of "false negatives" if the 
rules that dictate operation of the system are not 
continuously and timely updated. 

30 Additionally, the arrangement described herein 

provides system-wide operation based on anomaly 
detection, without limitations to any specific 
application and making it possible for the arrangement 
to become an expert system adapted to learn proper 

35 intervention policies. 
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Brief description of the annexed drawings 

The invention will now be described, by way of 

example only, by referring to the enclosed figures of" 

drawing, wherein: 
5 ~ figure 1 is a block schematic representation of 

an analysis system as described herein, 

- figure 2 to 7 are functional representations of 
various parts of the system of figure 1, and 

figures 8 is an exemplary representation of 
10 possible operation of the system described herein . 

Detailed description of preferred embodiments of 
the invention 

A possible embodiment of the arrangement described 
herein is portrayed in figure 1 in the form of a host- 
15 based intrusion detection system (HIDS) comprised of 
three high-level logical components, namely: 

a system-wide information gathering component 
110 which intercepts low- level data from a host 
computer (not shown) , thus being arranged "straddling" 
20 a kernel space and the user space proper; low- level 
data comprises system calls, or system primitives, with 
their call and return parameters, and, information 
relative to system resources in use (e.g. file, socket, 
device . - ) ; 

25 - a detection component 12 0 which performs data 

analysis in order to reveal possible intrusions, thus 
representing the core of the HIDS; and 

- a management system 13 0 which shows so-called 
alerts to be described in greater detail in the 

30 following, logs them for off-line analysis, generates 
reports, and allows the administration and 
configuration of the whole system. 

The detection component 12 0 can be in turn divided 
into three logical sub- components with specific goals: 
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- a first sub- component 104 is a current stste 
module that maintains a real-time high-level model of 
the current state of the monitored host ; 

- a second sub -component 105 (in turn comprised of 
5 a plurality of modules 105a, 105b, 105c, 105d, and 105e 

to be described later), using a high-level model, 
learns the ^'good'' behavior of the system by recording 
all possible states reached in a '''regular" period of 
work; then it performs anomaly detection by revealing 
10 differences between the instantaneous state and the 
recorded ones; 

- a third sub-component 106 is an alerter module 
that receives and correlates the anomalies to decide if 
they can represent an intrusion and, in this case, 

15 emits an alert to the management system 13 0. 

In the currently preferred embodiment of the 
arrangement described herein, the three components are 
realized by several modules that interact sequentially, 

A device driver 101 in the component 110 
20 intercepts system primitives (briefly defined 
"syscalls" or "system calls" in the following) in 
kernel space and provides them to a syscall processor 
103 . 

A kernel information module 102 reads all the 
25 required information about the processes running, 
allowing the other modules to build an instantaneous 
snapshot of the system state. This information is used 
in the current state module 104 for initialization and, 
when needed, for maintaining synchronization with the 
30 system - 

The syscall processor 103 translates syscalls into 
higher level, OS independent, abstractions and forwards 
them to the current state module 104. 
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The current state module 104 is initialized with 
the information taken from the kernel info module 102. 
Subsequently it uses the system call sequence stream, 
as provided by the syscall processor 103, to remain 
5 synchronized with the host system. If a mismatch is 
detected, the current state module 104 is able to scan 
the real system again (through the interface provided 
by the kernel info module 102) . After updating the 
internal state, the syscall event or the re- 
10 synchronization event is forwarded to a set of 
knowledge base modules 105 provided in the system. 

The knowledge base (KB) modules 105a*, ■I05b, 105c, 
105d, and 105e (collectively designated 105) use the 
syscall and the resynchronization events in two 

15 different conditions; during the learning mode, each 
event updates the internal database and enhances the 
knowledge on the system behavior; in the analysis mode, 
the event is matched against the database and, if an 
anomaly is detected, a so-called led alert is sent to 

20 an Alerter module 106. 

A led alert is essentially a collection of uniform 
data that indicates the cause of the anomaly, where and 
when it was generated. 

The alerter 106 receives led alerts from all the 
25 knowledge base modules 105, and correlates them in 
order to obtain significant information about the 
possible intrusions. The resulting alerts are then sent 
to the management system 13 0. 

In the presently preferred embodiment, the alerter 
30 module 106 is comprised of a basic correlation 
module /mechanism (discussed in detail in the following) 
and a fuzzy-logic inference engine configured to 
aggregate independent alerts, so as to suggest what 
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kind of anomaly has been effectively detected by the 
underlying sensors . 

The management system 13 0 consists of two logical 
parts. One part, designated 107, displays e-.g. on a 
5 graphic console the alerts coming from the HIDS, shows 
the current state of the monitored hosts and can save 
all the alerts onto a local database to provide off- 
line analysis, such as trend and forensic analysis. The 
other part, designated 108, provides all the functions 
10 to monitor and to configure the work of every single 
part of the HIDS . 

Three components (i.e. the kernel info module 102, 
the device driver 101, and the syscall processor 103) - 
among the various elements shown - are system 
15 dependent, and have to be implemented for every 
different operating system; the other elements 
described are system- independent and are based on a 
logical abstraction of data taken form the underlying 
system. 

20 The device driver 101 is a system-dependent 

component which intercepts a subset of all possible 
system calls along with their return value and 
invocation parameters. 

As shown in figure 2, the device driver 101 runs 
25 in kernel space and, upon activation, saves the 
structure containing the addresses of the sub-routines 
corresponding to each system call 2 01 (i.e. in IMIX- 
like systems, the array syscall__table [syscall -ID]) and 
substitutes it with its own sub-routines 202. Each of 
30 these sub-routines runs the savedsub-routine, acting sa 
a wrapper. In case of success, logs the syscall data 
203 on a FIFO device 204. Data is produced as a byte 
stream of system-dependent information, which will be 
read and translated into a higher level - system 
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independent - abstraction by the syscall processor 103. 

By way of example, in an embodiment for a UNIX- 
type system, the subset of system calls with related 
parameters and abstraction translation is as- shown in 
5 Table 1 below . 



TABLE 1 



ID 


Common Name 


Return Value 


Parameters 


Translation 


1 


EXIT 






processExit () 


2 


FORK 


new PID 




processFork ( ) 


5 


OPEN 


new file 
descriptor 


file name 
opening mode 
file name 
length 


• processOpen ( ) 


6 


CLOSE 




file descriptor 


processClose () 


8 


CREAT 


new file 
descriptor 


file name 
opening mode 
file name 
length 


processOpen ( ) 


11 


EXECVE 




process name 

process 

arguments 


T— 1 / \ 

processExec () 


23 


SETUID 




new UID 


processSetuid ( ) 


41 


DUP 




old file 
descriptor 
new file 
descriptor 


processDup () 


42 


PIPE 


new file 
descriptor 




processOpen ( ) 

processOpen ( ) 


63 


DUP2 




old file 
descriptor 


processDup () 
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new file 
descriptor 




70 


SETREUID 




new UID 


processSetuid ( ) 


120 


CLONE 


new PID 




processFork ( ) 


164 


SETRESUID 




new UID 


processSetuid ( ) 


190 


VFORK 


new PID 




processFork ( ) 


203 


SETREUID32 




new UID 


processSetuid ( ) 


208 


SETRESUID32 




new UID 


processSetuid ( ) 


213 


SETUID32 




new UID 


processSetuid ( ) 


251 


SOCKET 


new socket 
descriptor 


socket data 
(domain, 

port, protocol) 


processSocket () 


252 


BIND 




socket 
descriptor 
(port) 


processBind ( ) 


253 


CONNECT 




socket 
descriptor 


processConnect () 


254 


LISTEN 




socket 
descriptor 


processListen ( ) 


255 


ACCEPT 


new socket 
descriptor 


socket data 
(domain, 

port , protocol ) 
old socket 
descriptor 


processAccept () 


258 


SOCKETPAIR 


new socket 
descriptor 
pair 


socket data 
(domain, 

port, protocol) 


processSocket () 
processSocket () 



It will be appreciated that the system calls 
listed in the foregoing comprise a set .grouping all the 
system primitives that either allocate (i.e. request) 
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or release one of the system resources. 

The device driver 101 does not provide syscall 
logging for the process that uses it; if this happened, 
an unstable loop would occur: in fact, for each syscall 
5 logged, the user-space layer would execute "a ' read () 
operation, that would cause another event to be posted 
in the FIFO; this sequence of events would eventually 
lead to resource exhaustion for the Device Driver, and 
the system would stop working. 

10 The kernel info module 102 is another system- 

dependent component which is able to^ reaiS.- information 
for all processes running on the monitored system. It 
saves their status in internal variables and exposes 
public methods for accessing this information. 

15 In an exemplary embodiment for a UNIX- type system, 

as shown in figure 3, all the information needed is 
available from a /proc directory 301. The /proc 
directory contains a sub-directory entry for every 
running process; the sub-directory is named after the 

20 current PID of the process: for example, a process with 
PID number 916, the corresponding sub-directory is 
/proc/916, designated 302 in figure 3. 

The subdirectory contains a set of text files and 
links (in particular cmdline, exe, status) providing 

25 detailed information about the process, such as PID 
(Process IDentif ier) , PPID (Parent Process IDentif ier) , 
EUID (Effective User IDentif ier) , executable pathname, 
command line of invocation. A sub-directory- fd (so 
/proc/916/f d, designated 303) contains links to file 

30 descriptors (files, pipes, devices and sockets) 
currently opened by the process. Additional detailed 
information about open sockets can be found in 
/proc/net, designated 304, in the following text files: 
raw, tcp, udp, unix, packet. The Kernel Info reads all 
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needed data when the snapshot () method is invoked (at 
305) and fills its internal data structures. It then 
provides all this information to the current state- 
module 104 through known tools, designated 306, which 
5 allow to enumerate the data (begin(), end()) and to get 
specific info about a user (getUserInf o (UID) ) , a 
process (getProcInf o (PID) ) and a file or a socket 
descriptor (getDescriptorInf o (PID, FD) ) . These tools 
provide the results in a structure suitable to be used 
10 by the current state module 104, as better described 
later. 

X 

The enumeration interface is the same used in the 
so-called Standard Template Library (STL) , which is 
enclosed in all common C++ modern implementations. 

15 In that way, the kernel info module 102 provides 

all the data needed for the current state module 104 to 
have a complete view of the initial state when the IDS 
is started. However, the whole operation can be invoked 
at any time by the current state module 104, when 

20 synchronization with the real state of the system is 
lost, for example because the user- space module cannot 
keep pace with the device driver 101. Finally, this 
module adds the benefit of decoupling the underlying 
operating system from the current state data. 

25 The syscall processor 103 reads the system call 

binary data from the FIFO queue (shown in phantom lines 
in the bottom right portion of figure 1) associated 
with the device driver 101 and translates them into a 
higher level syscall abstractions. Among those 

30 considered herein, the syscall processor 103 is the 
last system- dependent component of the arrangement 
described. 

In an exemplary embodiment for a UNIX-type system, 
as shown in figure 4, the records 401 on the FIFO have 
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a fixed size part with standard system call 
information: ID, PID, PPID, return value, first 
argument, and an extra -argument size. Then, depending 
on the value of the extra -argument size, some more 
5 bytes contain the extra argument of the system call. 
The syscall processor 103 reads all these bytes and 
fills a generic system- independent syscall structure 
402 with PID, PPID, UID, datal (the syscall return 
value) , data2 (the syscall first argument) , data3 (the 

10 syscall extra -argument size) values. Then it invokes 
the corresponding member function of the current state 
module 104, as shown in Table 1. If needed, the extra 
argument of the syscall can be found as an extra- 
argument of the ProcessXxO method, such as the name in 

15 case of a ProcessOpen ( ) designated 403 in figure 4. In 
order to maintain a sufficient degree of abstraction, 
similar system calls are mapped onto generic one. For 
example, the open () , create () , and the pipe() syscalls 
are all mapped onto the processOpen () . 

20 The current state module 104 represents the 

instantaneous state of the monitored system; this 
abstraction is provided by monitoring all processes 
running on the system (grouped by owner of the 
process) , and all file descriptors and socket 

25 descriptors used by each process. 

A currently preferred embodiment of such module 

data is, as shown in figure 5, a container of users 
501, indexed by UID (User IDentifier) , having one or 
more running processes. Each user record 502 contains 

30 the UID and a process container 503 grouping all the 
running processes for that particular user. Each record 
504 of this group contains the name of the running 
executable file, PID, PPID, EUID and a container of 
descriptors 505, indexed by FID (File Descriptor). The 

35 descriptor is a generic abstraction for files and 
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sockets, as it commonly happens in real UNIX-like 
systems. The descriptor for a file 506 is composed by 
the file name and the opening mode for the file- 
(ReadOnly, WriteOnly, ReadWrite, Append) . The 
5 descriptor for a socket 507 contains the ^.protocol 
address information (usually IP addresses and ports) 
and the socket state (Created, Bound, Listening, 
Connected, Disconnected) . During the initialization, 
the user container is filled with the initial system 
10 state taken from the kernel info snapshot. 

Then, each system call causes a ckange of the 
internal state of the system, which is reflected in the 
current state module 104 using the ""processXx ( ) " member 
functions. The current state module 104 provides one of 

15 these functions for each abstract system call (as shown 
in Table 1) and keeps track of this change by updating 
its internal variables so to be synchronized with the 
monitored system. The current state module 104 also 
holds a list of knowledge base module 105; the puarpose 

20 of these modules is to perform a specialized analysis 
on a sub- component of the system. In order to do so, 
the current state module 104 acts as a synchronization 
point for all the associated knowledge base modules 
105. Whenever the current state module 104 receives a 

25 syscall event or requests a re- synchronization, this 
event is forwarded sequentially to all the knowledge 
base modules 105 linked to the current state module 
104. These modules can obtain further information about 
the system state by querying the current state module 

30 104. The interface used to do such queries is the same 
used in the kernel info module 102 . 

The knowledge base modules 105 differs one from 
another for their different views of the system: some 
are wide and generic while others are '"focused" and 
35 specialized. They monitor different aspect of the 
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system, so they have different variables and they use 
different ways of counting or taking into accounts the 
events that occur on the system. 

However, they all share two different -modes of 
5 operation, also shown in the state diagram of figure 6. 
This diagram shows a number of stages or states typical 
of operation of the modules 105. 

In a learning stage 601, the module fills its 
internal data structures (the so called ''knowledge 

10 base"), by recording the initial state (with an 
initialize 0 function 602), all the changes caused by 
the^' syscalls (with processXxO functions 603), and 
sometimes again the instantaneous state (with a 
synchronize 0 function 603) in case of loss of 

15 synchronization with the current state module 104. The 
learning stage can be stopped (at 604) (with a 
finalize 0 function 605) and restarted^ (with an 
initialize 0 function 606) until the module is supposed 
to have collected a comprehensive view on the system 

20 behavior . 

A normalization 0 function 607 operates a 
transaction between the two main stages*. The module 
performs an off-line optimization of the knowledge base 
by pruning useless data, or translating them into a 

25 more compact form so to obtain a more efficient 
database and optimize the analysis stage. Therefore 
after normalization 608 the database is copied into a 
completely new one used for the analysis stage. To 
follow on learning the old one should be used, while a 

30 not -normalized knowledge base cannot be used in 
analysis stage. 

In an analysis stage 609, the module uses the 
normalized knowledge base to analyze the state of the 
running system. The normalized database is* now 



wo 2005/059720 



PCT/EP2003/014385 



consolidated and it never changes, but it is used to 
match against every change caused by the system calls 
(with processXxO functions 610) or re- synchronization 
with the system (with a synchronize ( ) function 610) . 
5 Detection operates by revealing differences jwith the 
observed state- Internally, the module may use 
different logic mechanisms to keep track of those 
differences. In the current embodiment, a counting 
mechanism is used to track all the occurrences of 

10 certain events, such as the number of processes that a 
certain user is executing. Obviously, other mechanism 
are used accordingly the specific logic ''. requested in 
the module. Whenever the module detects some 
differences with the recorded state, it sends to the 

15 alerter 106 a led alert, which is a structure with all 
possible information about the detected anomaly. 

As shown in figure 7 the led alert has a standard 
part CP comprised of timestamp, module name, syscall 
ID, PID, PPID, UID, which is common to all knowledge 
20 base modules 105. This information is used to locate 
the part of the system where the anomaly happened and, 
for the alerter 106, to aggregate led alerts coming 
from different knowledge base modules 105, as will be 
described later in this chapter. 

25 The remaining part SP of the led alert is 

knowledge base module dependent. This means that each 
knowledge base module 105 adds all needed information 
to describe in depth the anomaly. 

Possible examples include, but are not limited to, 
30 the name of the variable/s that caused the anomaly^ 
its/their current value/s and the range of values 
registered in the knowledge base. In addition to the 
current information taken from the current state module 
104, every knowledge base module 105 assigns a weight 
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to each led alert it emits, according to the ^^distance" 
of the anomaly from the regular behavior that is the 
state recorded during the learning phase and saved in 
the knowledge base. These weights will be used by the 
5 alerter 106 to correlate several led alerts and ■ obtain 
a user-level alert as described in the following. 

A presently preferred embodiment of the 
arrangement described herein uses the following 
knowledge base modules 105 : 

10 - KBUserTable: this is a main module, designated 

105a in figure 1, which maps accurately the information 
present in the current state module 104. This module 
gives a system wide view, without focusing on a 
specific aspect . The internal structure is similar to 

15 the one shown in figure 3 for the current state module 
104. The module 105a works mainly by aggregating 
information such as the name of the process, or which 
user is executing a given process. Moreover, using 
specific counters, the module 105a keeps track of the 

20 number of different instances of a specific 
application, the number of opened files and sockets. A 
led alert is emitted whenever a new object, (a new user, 
which has never been active in the system, or a new 
process, that has never been observed for a specific 

25 user and so on) appears on the system. Another led - 
alert is emitted whenever one of the counters for a 
specific object is exceeded (because of a creation or 
removal of the particular object) . Of course, the 
module 105a does not keep track of instantaneous and 

30 time-variant data (such as PID and PPID) , because they 
can be different for each run; 

- KBProcessTree : this module, designated 105b in 
figure 1, complements the previous module 105a, and 
takes into account the parent /child relationship, -which 
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is a common relationship among tasks in a modern 
operating system. The module 105b records this 
information in a tree-like structure. Only the process- 
name and the relative position in the tree are 
5 recorded, and common instances of a specific, process 
are aggregated using counters. This kind of module is 
able to detect whether a specific process originates 
any entities that have never been observed during the 
learning stage; this is a fairly common situation, when 

10 an intruder tries to exploit a system flaw and obtain 
an illegal access to the system (usually a remote 
shell) . A led alert is emitted every time some 
difference in the process tree structure is observed: 
new child processes appear or crucial nodes (always 

15 present during the learning stage) disappear. 
Otherwise, a led alert with a lower weight can be 
raised when an application exceeds its usual number of 
active instances; 

KBNetwork: this module, designated 105c in 

20 figure 1, monitors network activity; it analyzes 
running processes mapping their ^^network behavior" , 
that is the number and type of connection that each 
process uses during its life. Connections are 
aggregated on a per-process basis; moreover, the module 

25 105c discriminates between ^^generic" connection, which 
are usually bound to any free port in the system, and 
^^server" connection, which are used in a server to 
specify a well-known point of access to the service. 
The module 105c also keeps track of the ■ traffic 

30 patterns used by specific process; that is, the number 
of inbound/ outbound packets, the number of 
inbound/ outbound simultaneous connection and so on. A 
led alert is emitted whenever a change of network 
behavior is observed. The list of conditions that may 

35 lead to led alert emission include: the creation of a 
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new listening connection; the reception of an exceeding 
number of connection requests; the creation of a 
connection originating from an application that has 
never used the network, the reception _ or the 
5 transmission of an exceeding number of packers- for a 
specific connection that has already been observed; 

KBFilesystem: this module, designated 105d in 
figure 1, keeps track of all file-system related 
operations, on system-wide basis; particularly, it 

10 detects new mount /unmount operations, the appearance of 
unknown files in directory where there was. no activity 
at all during the learning stage; the access to 
specific core files (such as the kernel, the device- 
driver modules and the security related files) , the 

15 excessive creation or deletion of files in a reduced 
amount of time, where the exact meaning of ^^excessive" 
has been deducted during the learning stage, the 
excessive use of symbolic linking or the creation of 
links in the directory holding temporary files. In all 

20 this situations led alerts are emitted with the 
appropriate weight, according to the gravity of the 
unusual behavior. The module 105d can also combine the 
standard anomaly detection mechanism with few misuse 
detection techniques tailored to improve the overall 

25 efficiency. For example a specific security related 
directory or file can be set to be monitored in a 
deeper mode by recording more accurately all actions 
performed on it so to emit more precise led alerts. 

KBDevices: this module, designated 105e in 
30 figure 1, monitors the system for device-driver or 
other in-kernel module related issues. The module 105e 
builds a list of commonly used modules, and correlates 
the use of specific modules with certain specific 
processes- For example, USB-related modules are 
35 dynamically loaded in the system when a USB device is 
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attached to it. So some correlation is expected to 
exist between the loading of an USB device driver, and 
specific programs used to manipulate it. Whenever the- 
system detects a module that has never been used 
5 before, or detect a module which is used in conjunction 
with uncommon executable files (for example, a module 
that spawns a remote shell as a response to a 
specifically crafted packet) , a led alert is sent to 
the alerter module 106. The module 105e is fairly 
10 simple in the case of a current Unix embodiment, but 
can easily be extended and tailored to monitor other 
kernel-based components, such as the xouting core 
system or the firewall enclosed in the kernel. 

The alerter 106 processes and aggregates all the 
15 led alerts received (i.e. the anomalies detected) from 
the various knowledge base modules 105. Usually, single 
led alerts are not necessarily a sign of an intrusion, 
but may derive from the regular system execution, which 
has never been observed during the learning stage. 
20 However, if the alerter 106 receives several led alerts 
from independent modules, a user- lever alert is 
generated and sent e.g. to the management console. 

The alerter 106 works essentially in two ways. 

As a first task, the alerter 106 tries to 
25 aggregate different led alerts using some mathematical 
correlation technique. - 

If the sequences of led alerts loosely match a 
sequence of pre-conditions in the rule-base-, it is 
possible to identify a specific behavior and issue an 
30 alert which also gives some information about what is 
happening on the system. 

The basic correlation mechanism works on the 
shared field of the led alert structure. The alerter 
106 uses the following logic to aggregate different led 
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alerts - 

If it receives several led alerts for the same PID 
(Process IDentif ier) , these alerts are aggregated over 
time according the following formula: 



Eq. 



where Wi is the weight of the user level alert 
associated to the common stream of led alerts, when the 
i-th led alert is received; Ti is the time^ of reception 
of the i-th led alert, LAi is the weight associated to 
10 the i-th led alert and r is the time-decay constant. 

This formula basically means that the weight 
associated with each specific user-level alert is 
composed by the value of the weight at the previous 
alert emission time plus the current value modulated 

15 with an exponential decay factor; the exponential decay 
indicates that the importance of the alert decreases 
over time. The current weight value is sampled at time 
Ti+i, when the alert is effectively received and, if the 
value is greater than a given threshold, a user-level 

20 alert for that PID is generated; the indication of the 
alert criticality is proportional to the weight value. 

A substantially identical aggregation criterion is 
used for led alerts with a matching value for the UID 
(User IDentif ier) . 

25 Several alerts with the same values in the common 

fields (except the timestamp) are aggregated again and 
only a single user- level alert is generated. The weight 
associated to this alert is computed again using 
equation 1. 

30 The rule -based correlation mechanism uses led 

alerts as the input of a fuzzy logic expert system. It 
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is possible to map the different led alerts into 
different fuzzy sets (one for each different led 
alert) ; using these fuzzy sets, it is possible to 
construct some fuzzy logic rules that indicate a 
5 specific exploitation attempt. 

The following list defines some possible fuzzy 
sets adopted to map specific alerts: 

NewApp: (KBUserTable 105a) An application that 
executes a new application that has never been executed 
10 before. 

NewChild: (KBProcessTree 105b) an application has 
just forked a new child application that has never been 
observed . 

NewListeningConnection (KBNetwork 105c) : An 
15' application activates a listening network connection 
that has never been activated before - 

NewReadFile (KBFileSystem 105d) : An application 
opens a file (read mode) that has never been open 
before . 

20 NewWriteFile (KBFileSystem lOSd) : An application 

opens a file (write mode) that has never been open 
before . 

MaximumCountApp (KBUserTable 105a) : An application 
exceeds its maximum number of concurrent instances. 

25 MaximumTraf f ic (KBNetwork 105c) : a network 

connection exceeds its maximum observed bandwidth. 

MaximumConnectionRequest (KBNetwork 105c) : a 
specific listening connection receives an exceeding 
number of connection requests . 

30 The following list details some rules that can be 

used for this purpose: 

Rulel: IF (NewApp and NewListeningConnection) 
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THEN UserAlert ( 'Possible Bindshell detected') 

Rule2 : IF (NewChild and NewListeningConnection) 

THEN UserAlert ( 'Possible Bindshell detected') 

Rule3 : IF (NewWriteFile MATCHES Vetc/*')^. 

5 THEN UserAlert ( 'Tried to write a config file'') 

Rule4 : IF (NewApp and MaximumCountingApp) 

THEN UserAlert ( 'Possible Local Denial-of -Service 
or Resource Exhaustion Attempt') 

Rule5 : IF (MaximumConnect ionRetguest and 

10 MaximumTraf f ic) 

THEN UserAlert (' Possible Remote Denial-of -Service 
Attempt' ) 

Of course, this rule list is not intended to be 
comprehensive, but has the purpose to show some of the 
15 possible attacks that this system can detect. The use 
of a fuzzy-logic inference engine allow the system to 
analyze the led alert trying to understand what is the 
possible cause of the led alert events detected on the 
system. 

20 The report and logging part 107 of the management 

system 103, as shown in figure 8, consists essentially 
of a graphic console C which shows on the screen 801 
all the alerts 8 02 coming from the different monitored 
hosts 803. It also shows the state 804 of the monitored 

25 systems by getting information 8 05 about the current 
state of the knowledge base modules 105 in the -host. It 
is also provided a mechanism to archive alerts on a 
database to perform further off-line analysis- A human 
operator monitors the management system 13 0 and can 

30 take the appropriate count ermeasure to block the 
attacks and to enforce policies.- Moreover, an 
administration and configuration part 108 of the 
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management systems 13 0 allows to watch and to configure 
the behavior of every single part of the whole system. 

Of course, the arrangement described in detail in- 
the foregoing represents only one of a plurality of 
5 possible implementations of the invention - 

A number of changes can be easily contemplated in 
alternative embodiments . 

For instance the device driver 101 could be 
enhanced by intercepting other system calls in order to 
10 monitor parts of the system that now are not taken into 
account. The syscall processor 103 may therefore map 
this new events into existing abstraction iunction (see 
Table 1) , or other new 'ad hoc' abstraction functions 
should be created. 
15 Also, new knowledge base modules 105 can be 

designed and implemented, such as, e.g. : 

-i) a KBUserProf ile module: to profile the usual 
behavior of users logged on a shell - This could be done 
by recording several information, such as the type of 

20 console (local/virtual) and connection 

(telnet/ssh/other) , the launched processes along with 
time of execution and parameters of invocation. 
Moreover this module can support operation profiling 
taking into account the various behavior in different 

25 time slots (for example, the user activity is higher in 
business hours) ; 

- ii) a KBRegistry module to monitor e.g. Windows™ 
registry activity. This module would be specific for 
all the Windows™ Operating System that use the registry 
30 (Windows XP, 2 000, NT, 98, 95) . The module should 
record all operations (create/open/ write/ close/delete) 
made by different processes on registry keys and 
values . 

The management system 13 0 can provide some form of 
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feedback to the alerter 10 6, in terms of e.g. logging 
level and emission of alerts. Moreover, a human 
operator can be given the opportunity to request an- 
update of the specific knowledge base modules 105, 
5 whenever some alerts can be tracked to regular^^. behavior 
that has never been observed during the learning stage. 
In that way, the detection capability of the system can 
be updated and enhanced without having to run another 
learning stage. 

10 Consequently, without prejudice to the underlying 

principles of the ' invention, the details and the 
embodiments may vary, also appreciably, with reference 
to what has been described by way of example only, 
without departing from the scope of the invention as 

15 defined by the annexed claims-- 



